Random Optimization Methods

D. Efficiency. In deterministic schemes, a lot P d h d t . d h of computer time is spent in deciding where the ure ran om searc , a ap lve ran om searc , . . I t d I. t . I . th I next probe pomt should be. In random search Slmu a e annea lUg, gene IC a gon ms, evo u... t. t t . t . t. t. th procedures, thIs tIme IS saved, at the expense Ion s ra egles, nonparame nc es lma Ion me -. ods bandit problems simulation optimization perhaps of a few more probes. Accordmg to the clu;terin g methods P'robabilis tl.C ut t d' minimax criterion, random search is more effi, a omaaan . h d ... random restart. These are a sample of the ranclent t an any eter~lDlstl~ search. This sa~s dom S h th d th t . d . th O that random search IS best m the worst possIble earc me 0 s a are revlewe m IS . paper. The discussion focuses on computational clrcu~st.ances. Jarvis (1975) in his ~urvey claims issues as well as behavior in difficult optimizat?at .It IS one. of the best ~ethods m the worst t. bl sItuatIon possIble (granulanty, plateaus, holes, Ion pro ems. ... dlscontmulties, high dimensionality, multimodality) and perhaps the worst method in the best Books. situation (smoothness, continuity, low dimenThis survey starts with the mention of a few sionality, unimodality)". good books on the subject matter. These inE. Flexibility. Random methods fill the enclude Zhigljavsky (1991), Torn and Zilinskas tire gap between pure random search (which (1989), Aarts and Korst (1989), Van Laarhoven tota~ly ignores any ~r~vi~usly obtained inforand Aarts (1987), Holland (1975), Schwefel (1981), matlon) and determlDlstlc methods. In fact, Schwefel and Manner (199.1), Ackley (1987), many are geared towards efficient combinations Goldberg (1989), Ermoliev and Wets (1988), and of methods. Wasan (1969). F. Information extraction. During the optimization process, the information gathered can Ad f d h be used to guide the search; this is especially vantages 0 ran om searc . f .. use ul when globalmformatlon about the shape Below are some reasons why you may wish to of the function has to be extracted. look at random search algorithms more closely. G. Easily parallelizable. Many random A. Ease of programming. Simple easily unsearch procedures either totally igllore past inderstood programs, that can be implemented on formation, or proceed with a number of simultanearly any computer. neous searches or moving clouds of points, with B. Inexpensive realization. Many of the only an occasional need for communication bemethods require very simple storage and comtween the various components. This lends itself parison facilities. There is virtually no overhead, superbly to parallelization. so that the cost of a run is virtually borne only H. Insensitivity to noise. Function evalby the number of function evaluations. uations that are perturbed by noise affect the , C. Insensitivity to the criterion function. performance of random search algorithms much Convergence of most random search procedures less than that of deterministic algorithms. Rancan be guaranteed for any function, regardless of dom search is also ideally suited for multimodal its smoothness properties or its multimodality. stochastic optimization problems. The particular shape (granularity, discontinuity, I. A simple startpoint selection method. presence of holes or plateaus) of a function has Pure random search and some of its variants can virtually no effect on most random search procebe used as a method for the selection of a suitdures. able startpoint of a local search algorithm. It is